Skip to content

fix(examples): host the kinetic-type A-roll so the video actually renders#1799

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/kinetic-type-example-media-in-subcomposition
Jun 30, 2026
Merged

fix(examples): host the kinetic-type A-roll so the video actually renders#1799
miguel-heygen merged 2 commits into
mainfrom
fix/kinetic-type-example-media-in-subcomposition

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

Problem

The kinetic-type flagship example fails hyperframes lint with 8 errors, and its A-roll footage is not driven by the live runtime. The centerpiece error is media_in_subcomposition: the A-roll <video> was authored inside the main-graphics sub-composition <template>.

Root cause

The runtime only seeks and decodes media that is a direct child of the host root (index.html). Media nested in a sub-composition template is never seeked, so it renders blank in Studio preview and the <hyperframes-player> web component. The composition's server render only worked because the producer's compile step hoists nested media and auto-stamps timing, which masked the contract violation. The example also carried crossorigin (breaks preview), an untimed video (media_missing_data_start), three gsap_css_transform_conflict cases where a CSS translateY would be discarded by a GSAP y tween, and an unbundled Libre Baskerville family with no @font-face.

Fix (root cause, no suppressions)

  • Move the <video> out of compositions/main-graphics.html and into index.html as a direct child of #root (archetype B from composition-patterns), with data-start / data-duration / data-track-index and class="clip". Its per-scene opacity reveal now runs on the main timeline at global time, since a sub-composition timeline cannot reach host elements.
  • Make the sub-composition a transparent overlay shell (background: transparent) stacked above the host video, so the type renders over the footage.
  • Remove crossorigin; mark the audible footage with data-has-audio="true".
  • Replace the three CSS translateY values with fromTo tweens (carson-line-1, wes-text-top, wes-text-bottom) so GSAP no longer overwrites them.
  • Swap the unbundled Libre Baskerville for the bundled Playfair Display serif (an auto-resolved family) so the renderer supplies the Wes Anderson typography, and drop the redundant -apple-system token from the body font stack.

Verification

  • hyperframes lint registry/examples/kinetic-type: was 8 errors, now 0 errors (2 pre-existing warnings remain: google_fonts_import, composition_self_attribute_selector).
  • hyperframes validate registry/examples/kinetic-type: passes (exit 0, no console errors).
  • Draft render + ffmpeg frame extraction: the A-roll composites correctly under the Swiss panel (3.0s), the David Carson "No timeline" headline (10.0s, correctly positioned after the fromTo change), and the Vignelli outro (14.0s). The Wes Anderson scene (12.5s) shows the elegant Playfair Display italic serif with the footage hidden as intended. ffprobe confirms both h264 video and aac audio streams in the output. Before and after renders match visually, with the after version now contract-correct and lint-clean.

…ders

The A-roll <video> was authored inside the main-graphics sub-composition
template. The runtime only seeks and decodes media that is a direct child of
the host root, so the footage was never driven by the live runtime and renders
blank in Studio preview and the player (the producer's compile step happened to
hoist it, masking the contract violation in server renders). The example also
failed lint with eight errors.

Root cause and fixes:
- Move the <video> out of compositions/main-graphics.html and into index.html
  as a direct child of #root, with data-start/data-duration/data-track-index
  and class="clip". Its per-scene opacity reveal now runs on the main timeline
  at global time, since a sub-composition timeline cannot reach host elements.
- Make the sub-composition a transparent overlay shell (background: transparent)
  and stack it above the host video so the type renders over the footage.
- Drop the crossorigin attribute (plain displayed media never needs it) and mark
  the audible footage with data-has-audio="true".
- Replace the three CSS translateY values that GSAP would overwrite with
  fromTo tweens (carson-line-1, wes-text-top, wes-text-bottom).
- Swap the unbundled Libre Baskerville for the bundled Playfair Display serif so
  the renderer can supply the Wes Anderson typography, and drop the redundant
  -apple-system token from the body font stack.

lint reports 0 errors, validate passes, and frame extraction confirms the A-roll
composites correctly under every type scene.
@miguel-heygen miguel-heygen marked this pull request as ready for review June 30, 2026 17:04

@james-russo-rames-d-jusso james-russo-rames-d-jusso left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at HEAD 7f5d0794.

LGTM. This is the right fix — root-cause (media_in_subcomposition violation) addressed by moving the <video> to the host root, not by suppressing the lint rule or working around the contract. The accompanying cleanup (CSS translateYfromTo tweens, Libre Baskerville → bundled Playfair Display, removing crossorigin, adding data-has-audio) tracks the rest of the 8 lint errors cleanly. Result: 0 errors, validate passes, render verified.

A few things I want to confirm and one observation:

Questions

1. Sub-composition timeline → host-root element coupling

The script in compositions/main-graphics.html no longer touches #a-roll-video, and the main timeline in index.html now drives the reveal via tl.set("#a-roll-video", { opacity: 1 }, 2.2) etc. The reveal points (2.2s, 11.7s, 13.5s) line up with what the sub-composition's own timeline does internally. Good.

But: the sub-composition's data-start="0" matches the host's, so global time == scene-local time. If a future remix puts data-start="3" on the graphics layer (or hot-swaps to a different sub-comp with its own start offset), the reveal events at 0 / 2.2 / 11.7 / 13.5 on the main timeline would no longer align with the graphics' internal beats. Worth a comment on the main timeline saying "reveal points must move with #graphics-layer's data-start if it ever drifts from 0" — or leave it as a known footgun documented in the example header.

Not a blocker for an example PR; just flagging the coupling.

2. Composition pattern reference for future authors

The "media must be a direct child of the host root" rule is enforced by media_in_subcomposition lint and now demonstrated correctly here. Is there an examples/ or docs/composition-patterns.md that should reference this PR as the canonical "A-roll under graphics overlay" example? If yes, link it in the example's README / header comment so future authors copying the kinetic-type structure don't reinvent the violation.

(Other examples I grepped — vignelli, swiss-grid, play-mode, warm-grain — all already put <video> at the host root, so the pattern is consistent with this fix. No sibling examples needed parallel patching.)

3. crossorigin removal — verify no breakage in remote-asset CORS path

You removed crossorigin="anonymous" from the <video> because it "breaks preview." That makes the asset opt-out of CORS-aware fetch, which is fine for playback but means JS-based pixel introspection (canvas.drawImage / WebGL texture reads) would taint the canvas. The kinetic-type composition doesn't appear to do pixel introspection, but worth confirming: does the runtime's frame-extraction path use Canvas read-back on the video? If yes, dropping crossorigin would break it for this specific example.

If the runtime uses screenshot capture from Chromium directly (not Canvas read-back), this is a non-issue.

4. data-has-audio="true" — first time in any example?

Adding data-has-audio="true" is a contract signal. Quick check: does the existing test/snapshot infrastructure for examples assert anything about which examples have audio? If yes, this PR may need to update the expected fixtures. If no, this is a clean addition.

Nits

  • The comment block at the top of compositions/main-graphics.html (lines 8-14 in the diff) is the right call — explains why the video is no longer here. Could also briefly link to the lint rule code (media_in_subcomposition) so a future debugger lands on the rule rather than guessing.
  • data-track-index="0" for the A-roll and data-track-index="1" for the graphics overlay is consistent with z-index (0 below, 1 above). Clean.

What I didn't verify

  • I didn't run hyperframes lint or hyperframes validate locally; trusting the PR body's claim of 0 errors and exit 0.
  • I didn't reproduce the draft render to inspect the actual frame composites; trusting the ffmpeg/ffprobe verification narrative.
  • I didn't trace the Studio preview path to confirm crossorigin removal genuinely fixes preview without breaking any pixel-read code path.

Review by Rames D Jusso

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — fix(examples): host the kinetic-type A-roll so the video actually renders

Verdict: 🟢 LGTM with nits
Repo: heygen-com/hyperframes Head: 7f5d079
Scope: Move A-roll <video> out of the main-graphics sub-composition into index.html root, refactor the per-scene opacity reveal to the main timeline, swap three CSS transform: translateY initials for GSAP fromTo, swap unbundled Libre Baskerville for bundled Playfair Display, drop redundant -apple-system token.

Summary

Clean root-cause fix that respects the renderer contract (media must be a direct host-root child to be seeked/decoded). Sub-composition becomes a transparent overlay shell, the A-roll lives in index.html, and its reveal is driven on the main timeline at global time — which is correct because the sub-comp data-start="0" means scene-local time equals global time. CSS-vs-GSAP transform conflict resolved by switching to fromTo on the three offending elements. Font swap to a bundled family is also correct. Example-only blast radius; LGTM with a couple of watchpoints around pre-tween scene timing and display=block.

Findings

💭 fromto-pre-tween-pose-gap

File: registry/examples/kinetic-type/compositions/main-graphics.html:393-457 (CSS) and the tl.fromTo(...) calls at the --- Scene 4 / --- Scene 5 blocks

Before this PR, the three elements had static CSS initial positions:

  • .carson-line-1transform: translateY(200px)
  • .wes-text-toptransform: translateY(-100px)
  • .wes-text-bottomtransform: translateY(100px)

…and the tween was tl.to(".carson-line-1", { y: 0, ... }) — the static CSS held the pre-state.

After this PR, the static CSS translateY is removed and the tween becomes tl.fromTo(".carson-line-1", { y: 200 }, { y: 0, ... }, 9.7).

For a paused timeline driven by seek(t):

  • At t < 9.7, GSAP has not yet applied any y to .carson-line-1, so its computed transform is the CSS default → y = 0 (its final intended position).
  • #scene-4 becomes visible at t=9.6 (tl.to("#scene-4", { opacity: 1, duration: 0.1 }, 9.6)).
  • For the 0.1s window from 9.6→9.7, the scene is fading in with .carson-line-1 already at its final y=0 instead of pre-rolling from y=200.

Same hazard for .wes-text-top / .wes-text-bottom between t=11.7 (scene-5 fade-in start) and t=12.0 / t=12.6 (the fromTo tween start times) — a 0.3s and 0.9s window respectively, though .scene opacity is also ramping over 0.1s so most of the gap is invisible.

If draft renders show the intended dramatic effect (recon brief and PR notes both say they do), the renderer's seek behavior is likely auto-applying fromTo start states ahead of t. But the safe shape is tl.set(".carson-line-1", { y: 200 }, 0); tl.to(".carson-line-1", { y: 0, duration: 0.6, ease: "expo.out" }, 9.7); — explicit pre-pose at t=0 leaves no ambiguity. Same for the two wes-text elements. Not blocking — the visual result reportedly looks right — but the explicit set at t=0 is the more defensible pattern, and matches the structure used for #a-roll-video in index.html (line 85: tl.set("#a-roll-video", { opacity: 0 }, 0)).

🟠 font-display-block-headless-render

File: registry/examples/kinetic-type/index.html:11

<link href="https://fonts.googleapis.com/css2?family=Inter:ital,wght@0,400;0,700;0,900;1,400&family=Playfair+Display:ital,wght@0,400;0,700;1,400&display=block" rel="stylesheet" />

display=block blocks text rendering until the font loads ("invisible text for up to 3s, then swap"). In a headless-Chrome render with a cold cache, the first ~1-3s of frames can show empty <div>s where the typography should be. The PR notes confirm visual correctness at 12.5s; that's well past the font-load window. But anything that lands in the first ~2s (Scene 1 "What if you could just" hook text at t=0.2) is at risk.

Two paths:

  • display=swap — fallback font renders immediately, swaps to the real face when ready. Brief FOUT (style change mid-render) is the cost; on a 15s timeline at 30fps the first ~30-90 frames might use the fallback. Probably acceptable here because the Inter fallback is also sans-serif.
  • Pre-warm in the renderer / await document.fonts.ready before the first frame. The renderer might already do this — if so, block is fine.

If the renderer already gates first-frame capture on document.fonts.ready, leave block alone and add a one-line HTML comment saying so. Otherwise prefer swap. Mid-bar concern — this is a flagship example, and a flagship example silently FOIT-ing on a cold-cache render would be a bad first impression.

💭 selector-scope-shift

File: registry/examples/kinetic-type/compositions/main-graphics.html (timeline body) and registry/examples/kinetic-type/index.html

The old timeline had const video = document.getElementById("a-roll-video") and operated on the video by reference. Now the video lives in index.html and the sub-comp timeline no longer touches it. Walked the remaining selectors in main-graphics.html's timeline:

  • #scene-1, #scene-2, ..., #scene-6, #scene-6-overlay, #white-wipe, .hook-what-if, .swiss-title, .swiss-subtitle, .bauhaus-shape-*, .carson-line-1, .carson-line-2, .carson-container, .wes-text-top, .wes-rule, .wes-text-bottom, .minimalist-bar, .minimalist-text, etc.

All of these resolve within the sub-comp's template — none cross into the host root. No scope-shift hazard. The only host-root element the old timeline reached (a-roll-video) is removed cleanly. Good.

🟢 host-root-archetype-correctness

The shape — host-root <video class="clip"> with data-start / data-duration / data-track-index / data-has-audio / playsinline, and the sub-comp data-composition-id / data-composition-src sibling at z-index: 1 — matches the "archetype B" pattern from the composition-patterns guide referenced in the PR description. object-fit: cover, z-index: 0, opacity: 0 initial, opacity tween on the main timeline — all clean.

🟢 crossorigin-removed

Old crossorigin="anonymous" on the <video> is gone. The PR description notes this breaks Studio preview. The new <video> correctly omits it; data-has-audio="true" is the right declarative replacement for the audio track.

🟢 transparent-overlay-stacking

main-graphics.html:151 flips background: blackbackground: transparent for the [data-composition-id="main-graphics"] selector, and the sub-comp wrapper in index.html:74 gets z-index: 1 while the host-root video gets z-index: 0. Stacking order is right; the graphics paint over the footage.

💭 font-stack-drop-apple-system

File: registry/examples/kinetic-type/index.html:27

-apple-system removed from the body font-family stack. Inter is the only family-with-name now plus the generic sans-serif. Reasonable cleanup — -apple-system was a leftover from a system-fallback aesthetic that doesn't fit the bundled-Inter design intent. If the renderer is a headless Chrome on Linux, -apple-system was a no-op anyway. No issue.

Verification

  • Read both files at head 7f5d0794fb1b3c9e4e447ad170eb62f10eaec0d2 via gh api .../contents.
  • Read full diff (additions=52, deletions=37, changed_files=2) — confirmed scope is exactly the two example files, no engine / runtime touch.
  • Walked the remaining sub-comp timeline selectors against the sub-comp DOM to confirm no host-root references survive in main-graphics.html.
  • Confirmed .scene base class has opacity: 0; scene-4 and scene-5 are invisible at t=0 (relevant to the fromTo-pre-tween-pose nit above).
  • Convergent with Rames (LGTM): he flagged separate concerns — data-start coupling between the main-timeline reveal points and #graphics-layer, CORS / canvas-tainting risk from dropping crossorigin, and possible test-fixture impact from adding data-has-audio="true". All complementary to my watchpoints (fromTo pre-tween pose gap, display=block headless behavior, font-stack cleanup). No contradiction.
  • Pre-post freshness check immediately before submit.

Review by Via

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved — independently confirmed CI green at HEAD and the change matches its described scope; converges with RDJ + Via's LGTM (their detailed reviews stand). — Rames Jusso

@james-russo-rames-d-jusso james-russo-rames-d-jusso left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R2 verification. R1 SHA 7f5d0794 → R2 SHA 4689cd46 (+6/-0 in one file).

✅ Net: coupling-doc concern resolved via explicit COUPLING comment. Other R1 questions remain open (non-blocking).

Verified

✅ Sub-composition timeline → host-root coupling — resolved via documentation (R1 Questions/1)

index.html:85-90 adds an 8-line COUPLING comment immediately before the tl.set/tl.to calls on #a-roll-video:

// COUPLING: the reveal/hide times below (2.2 / 11.7 / 13.5) are hand-tuned
// to the graphics-layer sub-composition's scene boundaries and assume it is
// mounted at data-start="0". If you re-time the sub-composition's scenes or
// change its data-start, these absolute times drift out of alignment and
// must be updated to match the new scene boundaries.

This names the exact failure mode I flagged in R1: if a future remix changes data-start or re-times the sub-composition's scenes, the absolute reveal times here drift. Legitimate close via intent-documentation — a future author copying this archetype will land on the warning before changing either side. When the concern is "this isn't enforceable in code but the next author needs to know", a load-bearing comment is the right shape.

Open / not addressed (non-blocking)

  • crossorigin removal — pixel introspection check (R1 Questions/3). Not addressed in delta. If the runtime's screenshot path uses Chromium-direct capture (not Canvas read-back), this is a non-issue — but unverified. Worth a one-line confirm from a runtime owner.
  • data-has-audio="true" test/fixture impact (R1 Questions/4). Not addressed. If no fixture asserts the audio-bearing example set, this is a clean addition.
  • fromTo pre-tween pose gap (Via's fromto-pre-tween-pose-gap). Not addressed. Via flagged a 0.1s–0.9s window where .carson-line-1 / .wes-text-top / .wes-text-bottom may render at their final-y position (instead of pre-rolling from the offset) during the scene fade-in. Drafts reportedly look right, so the GSAP seek behavior may auto-apply fromTo start states. Optional belt-and-suspenders: tl.set(".carson-line-1", { y: 200 }, 0) at t=0 to pin the pre-pose explicitly, matching the tl.set("#a-roll-video", { opacity: 0 }, 0) pattern at index.html:91.
  • display=block headless render (Via's font-display-block-headless-render). Not addressed. Could surface as FOIT on a cold-cache render for Scene 1's t=0.2 hook text. Probably handled if the renderer gates first-frame capture on document.fonts.ready; otherwise prefer display=swap.
  • Cross-example pattern reference (R1 Questions/2). Not addressed. Worth linking the canonical archetype-B pattern doc from the example header.

What I didn't verify

  • Didn't run hyperframes lint or hyperframes validate locally; trusting the PR body's claim of 0 errors and exit 0 (unchanged at R2).
  • Didn't reproduce the draft render to inspect frame composites.

Review by Rames D Jusso

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving at the new head — the data-start coupling comment landed (RDJ R2 confirmed). — Rames Jusso

@miguel-heygen miguel-heygen merged commit c01d1ae into main Jun 30, 2026
52 checks passed
@miguel-heygen miguel-heygen deleted the fix/kinetic-type-example-media-in-subcomposition branch June 30, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants